We previously showed that histone variant H2A.Z is critical for the self-renewal and differentiation by maintaining an open chromatin structure at key regulatory regions of transcription. It is also known that pluripotency of embryonic stem (ES) cells is controlled in part by chromatin-modifying factors that regulate histone H3 lysine 4 (H3K4) methylation. However, it remains unclear how H3K4 demethylation contributes to ES cell function. We showed that KDM5B, which demethylates lysine 4 of histone H3, co-localizes with H3K4me3 near promoters and enhancers of active genes in ES cells; its depletion leads to spreading of H3K4 methylation into gene bodies and enhancer shores, indicating that KDM5B functions to focus H3K4 methylation at promoters and enhancers. Spreading of H3K4 methylation to gene bodies and enhancer shores leads to differential gene expression during self-renewal and differentiation of Kdm5b depleted ES cells, and decreased enhancer activity, as measured by H3K27ac levels. KDM5B critically regulates H3K4 methylation at bivalent genes during differentiation in the absence of LIF or Oct4. We also show that KDM5B and LSD1, another H3K4 demethylase, co-regulate H3K4 methylation at active promoters but they retain distinct roles in demethylating gene body regions and bivalent genes. Our results provide global and functional insight into the role of KDM5B in regulating H3K4 methylation marks near promoters, gene bodies, and enhancers in ES cells and during differentiation.